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Abstract. A new framework for formulating reachability problems with competing inputs, 
nonlinear dynamics and state constraints as optimal control problems is developed. Such 
reach-avoid problems arise in, among others, the study of safety problems in hybrid systems. 
Earlier approaches to reach- avoid computations are either restricted to linear systems, or face 
numerical difficulties due to possible discontinuities in the Hamiltonian of the optimal control 
problem. The main advantage of the approach proposed in this paper is that it can be applied 
to a general class of target hitting continuous dynamic games with nonlinear dynamics, and 
has very good properties in terms of its numerical solution, since the value function and the 
Hamiltonian of the system are both continuous. The performance of the proposed method is 
demonstrated by applying it to a two aircraft collision avoidance scenario under target window 
constraints and in the presence of wind disturbance. Target Windows are a novel concept in 
air traffic management, and represent spatial and temporal constraints, that the aircraft have 
to respect to meet their schedule. 

1. Introduction 

Reachability for continuous and hybrid systems has been an important topic of research in the 
dynamics and control literature. Numerous problems regarding safety of air traffic management 
systems [1], [2], flight control [3], [4], [5] ground transportation systems [6], [7], etc. have been 
formulated in the framework of reachability theory. In most of these applications the main aim 
was to design suitable controllers to steer or keep the state of the system in a " safe" part of the 
state space. The synthesis of such safe controllers for hybrid systems relies on the ability to solve 
target problems for the case where state constraints are also present. The sets that represent the 
solution to those problems are known as capture basins [8]. One direct way of computing these 
sets was proposed in [9], [10], and was formulated in the context of viability theory [8]. Following 
the same approach, the authors of [11], [12] formulated viability, invariance and pursuit-evasion 
gaming problems for hybrid systems and used non-smooth analysis tools to characterize their 
solutions. Computational tools to support this approach have been already developed by [13]. 

An alternative, indirect way of characterizing such problems is through the level sets of the 
value function of an appropriate optimal control problem. By using dynamic programming, 
for reachability /invariant /viability problems without state constraints, the value function can 
be characterized as the viscosity solution to a ffist order partial differential equation in the 
standard Hamilton- Jacobi form [14], [15], and [16]. Numerical algorithms based on level set 
methods have been developed by [17], [18], have been coded in efficient computational tools by 
[16], [19] and can be directly applied to reachability computations. 

In the case where state constraints are also present, this target hitting problem is the solution 
to a reach-avoid problem in the sense of [1]. The authors of [1], [20] developed a reach-avoid 
computation, whose value function was characterized as a solution to a pair of coupled variational 
inequalities. In [19], [21], [22] the authors proposed another characterization, which involved only 
one Hamilton- Jacobi type partial differential equation together with an inequality constraint. 
These methods are hampered from a numerical computation point of view by the fact that the 
Hamiltonian of the system is in general discontinuous [20]. 
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In [23], a scheme based on ellipsoidal techniques so as to compute reachable sets for control 
systems with constraints on the state was proposed. This approach was restricted to the class 
of linear systems. In [24], this approach was extended to a list of interesting target problems 
with state constraints. The calculation of a solution to the equations proposed in [23], [24] is 
in general not easy apart from the case of linear systems, where duality techniques of convex 
analysis can be used. 

In this paper we propose a new framework of characterizing reach-avoid sets of nonlinear 
control systems as the solution to an optimal control problem. We consider the case where 
we have competing inputs and hence adopt the gaming formulation proposed in [15]. We first 
restrict our attention to a specific reach-avoid scenario, where the objective of the control input 
is to make the states of the system hit the target at the end of our time horizon and without 
violating the state constraints, while the disturbance input tries to steer the trajectories of the 
system away from the target. We then generalize our approach to the case where the controller 
aims to steer the system towards the target not necessarily at the terminal, but at some time 
within the specified time horizon. Both problems could be treated as pursuit-evasion games, and 
for a worst case setting we define a value function similar to [24] and prove that it is the unique 
continuous viscosity solution to a quasi- variational inequality of a form similar to [25], [26]. The 
advantage of this approach is that the properties of the value function and the Hamiltonian (both 
of them are continuous) enable us using existing tools to compute the solution of the problem 
numerically. 

To illustrate our approach, we consider a reach-avoid problem that arises in the area of 
air traffic management, in particular the problem of collision avoidance in the presence of 4D 
constraints, called Target Windows. Target Windows (TW) are spatial and temporal constraints 
and form the basis of the CATS research project [27], whose aim is to increase punctuality and 
predictability during the ffight. In [28] a reachability approach of encoding TW constraints was 
proposed. We adopt this framework and consider a multi-agent setting, where each aircraft 
should respect its TW constraints while avoiding conffict with other aircraft in the presence 
of wind. Since both control and disturbance inputs (in our case the wind) are present, this 
problem can be treated as a pursuit-evasion differential game with state constraints, which are 
determined dynamically by performing conffict detection. 

In Section II we pose two reach-avoid problems for continuous systems with competing inputs 
and state constraints, and formulate them in the optimal control framework. Section III provides 
the characterization of the value functions of these problems as the viscosity solution to two 
variational inequalities. In Section IV we present an application of this approach to a two 
aircraft collision avoidance scenario with realistic data. Finally, in Section V we provide some 
concluding remarks and directions for future work. 

2. Differential games and Reach- Avoid problems 

2.1. Differential game problem formulation. Consider the continuous time control system 
X = /(x, ix, v)^ and an arbitrary time horizon T > 0. with x eW^, u e U C R^, v e VCW^ and 
/(•, •, •) : R"' X /7 X V ^ R^. Let ^t,t']7 ^[t,^'] denote the set of Lebesgue measurable functions 
from the interval [t^f] to /7, and F respectively. Consider also two functions /(•) : R"^ ^ R, 
/i(-) : R"^ ^ R to be used to encode the target and state constraints respectively. 

Assumption 1. /7 C R^ and F C R^ are compact, f , I and h are hounded and Lipschitz 
continuous in x and continuous in u and v. 

Under Assumption 1 the system admits a unique solution x(-) : [t,T] for all t G [0,T], 

u{-) e V{[t,T] and v{-) G V[t,T]- For r G [t,T] this solution will be denoted as 

(1) (/)(r,t,x,2x(-), = x{t). 

Let C/ > be a bound such that for all x^x eW^ and u{-) G ^^,T] and for all u e 

\f{x^u)\ < Cf and u) — f{x^u)\ < Cf\x — x\. 
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Let also Ci > and Ch > he such that 

\l{x)\ < Ci and \l{x) - l{x)\ < Ci\x - x\, 
\h{x)\ < Ch and \h{x) - h{x)\ < Ch\x - x\. 

In a game setting it is essential to define the information patterns that the two players use. Fol- 
lowing [29], [15] we restrict the first player to play non-anticipative strategies. A non-anticipative 
strategy is a function 7 : V[o,t] ^ ^o,T] such that for all s G [t,T] and for all v^v G V, if 
v{r) = v{r) for almost every r G [t,s], then 'y[v]{r) = 7['0](r) for almost every r G [t^s]. We 
then use ^[t,T] to denote the class of non-anticipative strategies. 

Consider the sets A related to the level sets of the two bounded, Lipschitz continuous 
functions / : R and h : R^ R respectively. For technical purposes assume that R is 

closed whereas A is open. Then R and A could be characterized as 

= {x G R^ I l{x) < 0}, A = {xe R^ I h{x) > 0}. 



2.2. Reach- Avoid at the terminal time. Consider now a closed set R C R^ that we would 
like to reach while avoiding an open set A C R^. One would like to characterize the set of 
the initial states from which trajectories can start and reach the set R at the terminal time 
T without passing through the set A over the time horizon [t,T]. To answer this question on 
needs to determine whether there exists a choice of 7 G ^[t,T] such that for all v{-) G V[t^T]^ the 
trajectory x{-) satisfies x{T) G R and x{r) G A^ for all r G [t,T]. 
The set of initial conditions that have this property is then 

(2) RA{t,R,A) = {xeR^ I 37(-) G rf,,^], V^(-) G V[,,t], 

((/)(T,t,x,7(-),^(-)) e ^) A (Vr G [t,T], (/)(r, t, x, 7(-), ^(O) ^ ^)}. 
Now introduce the value function 1/ : R^ x [0, T] ^ R 

(3) V{x^t) = inf sup max{/((/)(T, t, X, ix(-), 'L'(-))), max h{<p{r^t^x^u{-)^v {■)))}. 

V can be thought of as the value function of a differential game, where u is trying to minimize, 
whereas v is trying to maximize the maximum between the value attained by / at the end T 
of the time horizon and the maximum value attained by h along the state trajectory over the 
horizon [t,T]. Based on [14], [15] and [25], we will show that the value function defined by (3) 
is the unique viscosity solution of the following quasi- variational inequality. 

dV dV 

(4) max{/i(x) - V(x,t), -^{x,t) + sup mf—-{x,t)f{x,u,v)} = 0, 

ot yue u ox 

with terminal condition V(x,T) = max{/(x), 

It is then easy to link the set RA{t^ R, A) of (2) to the level set of the value function V{x^ t) 
defined in (3). 



Proposition 1. RA{t,R,A) = {x G R^ | V{x,t) < 0}. 



Proof. V{x,t) < if and only if inf^(.)^r[,,:r] s^P^(-)eV[t,T] "^ax{/(^(T, t, x, 7(-), ^lO)), 
max^^[^^7^] /i(^(r, t, X, 7(-), 'u(-)))} < 0. Equivalently, there exists a strategy 7(-) G ^[t^r] such 
that for ah v{-) G V[^,t], max{/(0(T, t, x, 7(-), ^(•))), max^^[^,T] ^(^(^, ^, ^, 7(-)^ ^(O))} < 0- The 
last statement is equivalent to there exists a 7(-) G ^[t^r] such that for all v{-) G V[t^T]i 
/(^(T, t, X, 7(-), 'u(-))) < and max^^[^^7^] /i(^(r, t, x, 7(-), 'u(-))) < 0. Or in other words, there 
exists a 7(-) G ^[t^r] such that for all v{-) G V[t^T]i <P{T^t^x^^{-)^v{-)) G R and for all r G 
[t,T] (/)(r,t,x,7(-),^(-)) ' □ 
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2.3. Reach- Avoid at any time. Another related problem that one might need to characterize 
is the set of initial states from which trajectories can start, and for any disturbance input can 
reach the set R not at the terminal, but at some time within the time horizon [t, T], and without 
passing through the set A until they hit R. In other words, we would like to determine the set 

(5) RA{t,R,A) = {xeR^ I 37(-) G r^t,T]. M') ^ V[,,t], 

3ri G [t,T], ((/)(ri,t,x,7(.),^(-)) ^ ^) A (Vr2 G [t,ri], (/)(r2, t, x, 7(-), ^(O) ^ A)}. 
Based on [30], define the augmented input diS u = [u u] G Ux [0, 1] and consider the dynamics 

(6) f{x,u,v) = uf{x,u,v). 

Let (^(r, X, t, 'Lt(-), 'L'(-)) denote the solution of the augmented system, and define [/, U and T 
similarly to the previous case. Following [30] for every u G ^^,T] the pseudo-time variable 
a: [t,T] ^ [t,T] is given by 

(7) cr(r) = ^ + ^ u{s)ds. 

Consider a* to be almost an inverse of a in the sense that g{(j*{t)) = r. In [30], a* was defined 
as the limit of a convergent sequence of functions, and it was shown that 

(8) (/)(a(r),x,t,^a*(-)),^(a*(-))) = ^(r, x, t, ^(O), 

for any r G [t,T]. Based on the analysis of [30], equation (8) implies that the trajectory (p of the 
augmented system visits only the subset of the states visited by the trajectory (j) of the original 
system in the time interval [t,(j(r)]. 
Define now the value function 

V{x^t)= inf sup max{/((^(T, t, X, 7['u](-), 'u(-))), max /i(^(r, t, x, 7['u](-), 'u(-)))}. 

One can then show that V is related to the set RA. 

Proposition 2. For t G [0,T], RA{t,R,A) = {x \ V{x,t) < 0}. 

The proof of this proposition is given in Appendix A. 

3. Characterization of the value function 

3.1. Basic properties of V. We first establish the consequences of the principle of optimality 
for V. 

Lemma 1. For all (x, t) x [0, T] and all a G [0, T - t]: 

(9) 

Vix^t)= inf sup [maxj max /i(^(r, t, x, + a, t, x, -uf-)), t + . 

Moreover, for all {x,t) G x [0,T], V{x,t) > h{x) . 

The proof for the second part is straightforward and follows from the definition of V. The 
proof for the first part is given in Appendix B. 

We now show that F is a bounded, Lipschitz continuous function. 

Lemma 2. There exists a constant C > such that for all (x, t), (x^t) G x [0,T].' 

\V{x,t)\ < C and \V{x,t) -V{x,i)\ < C{\x - x\ + \t-i\). 
The proof of this Lemma is given in Appendix B. 
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3.2. Variational inequality for V. We now introduce the Hamiltonian H : x ^ R, 
defined by 

H{p^x) = sup inf p^f{x^u). 

Lemma 3. There exists a constant C > such that for all p^q eW^, and all x^y eW^: 

\H{p,x)-H{q,x)\ < C\p-q\ and \H{p, x) - H{p,y)\ < C\p\\x - y\. 

The proof of this fact is straightforward (see [14], [15] for details). We are now in a position 
to state and prove the following Theorem, which is the main result of this section. 

Theorem 1. V is the unique viscosity solution over {x^t) G R^ x [0,T] of the variational 
inequality 

dV dV 
mdix{h(x) — V(x^ t), (x, t) + sup inf (x, t)f(x, v)} = 0, 

Ot ^^ynGt/ ox 

with terminal condition V{x^T) = max{/(x), 

Proof Uniqueness follows from Lemma 2, Lemma 3 and [25]. Note also that by definition of the 
value function we have V(x,T) = max{/(x), h{x)}. Therefore it suffices to show that 

(1) For ah (xo.to) G R^ x (0,T) and for ah smooth W : R^ ^ R, if V - W attains a local 
maximum at (xo,to), then 

dW dW 
max{/i(xo) - V(xo,to), ^r-(^o,^o) + sup inf -7^{xo,to)f{xo,u,v)} > 
ot y^yueu ox 

(2) For ah (xo.to) G R^ x (0,T) and for ah smooth W : R^ ^ R, if V - W attains a local 
minimum at (xo,to), then 

dW dW 
max{/i(xo) - V{xo,to), ^— (xo,to) + sup inf -7^{xo,to)f{xo,u,v)} < 
ot y^yueu ox 

The case t = is automatically captured by [31]. 

Part 1. Consider an arbitrary (xo.to) G R^ x (0,T) and a smooth W : R^ x (0,T) ^ R 
such that V — W has a local maximum at (xo,to). Then, there exists Si > such that for all 
{x,t) gR^ X (0,T) with |x-xoP + (t-to)^ < 

{V-W){xo^to)>{V-W){x,t). 

We would like to show that 

dW dW 
max{/i(xo) - V(xo,to), ^-(xo,to) + sup inf — — (xq, to)/(a:o, ^, ^)} > 0. 
ot y^yueu ox 

Since by Lemma 1 h{x) — V(x,t) < 0, either h{xo) = V{xo,to) or, h{xo) — V(xo,to) < 0. For 
the former the claim holds, whereas for the latter it suffices to show that there exists v ^ V such 
that for all 1^ G U 

dW dW 

-g^{xo,to) + -^{xo,to)f{xo,u,v) > 0. 

For the sake of contradiction assume that for all v G V there exists u G U such that for some 
(9 > 

dW dW 

— (xo,to) + —{xo,to)f{xo,u,v) < -20 < 0. 
Since W is smooth and / is continuous, then based on [15] we have that 

-g^[Xo,to) + -^{xo,to)f[xo,u,Q < - y < 0, 

for all C G B{v^ r) H F and some r > 0, where B{v^ r) denotes a ball centered at v with radius r. 
Because Vis compact there exist finitely many distinct points ^i, ^ 'Wi, ...,'Un ^ and 
ri,...,rn > such that Fc Ur=i ^('^«' ^0 C ^ B{vi^ri) 

dW, , dW, 3(9 ^ 

-^[xo.to) + -^[xo,to)f[xo,UiX) < -y < 0. 



6 K. MARGELLOS AND J. LYGEROS 

Define g : Uhj setting for A: = 1, ...,n, g{v) = Uk v E B{uk,rk)\[j^Zi B{ui,ri). Then 

dW , , dW , , , , 3(9 ^ 

-^{xo.to) + -^{xo,to)f{xo,g{v),v) < -y < 0. 

Since W is smooth and / is continuous, there exists 62 G (0, Si) such that for all (x, t) G x (0, T) 
with \x - + (t - to)^ < ^2 

+ -g^{x,t)f{x,g{v),v) <-0<0. 

Finally, define 7 : V[to,T] ^ ^^o,T] by j[v]{t) = g{v{r)) for ah r G [to,T]. It is easy to 
see that 7 is now non-anticipative and hence 7(-) G T^t^^x]- So for all v{-) G l^[to,T] all 
G M'' X (0,T) such that \x - xo\^ + - ^0)^ < ^2, 

-gfi^^t) + ^(^,t)/(x,7M(-),^(-)) < < 0. 

By continuity, there exists (^3 > such that \(l){t,to, xo,j[v]{-),v{-)) — xop + (t — to)^ < ^2 for all 
t G [to, to + ^3]- Therefore, for all v{-) G V^tQ^r] 

V{4^{t, to, xo, 7(-), ^(O), ^) - ^(^0, to) < W{^{t, to, xo, 7(-), ^(O), ^) - W{xo, to) 
= / [-g^{(t>{s,to,xo,^{'),v{')),s) 

dW \ 
+ ^(0(5,to,xo,7(-)^'^(-))^^)/(0('5^^o,^o,7(-)^'^(-))^7(-)^'^(-))j^^^ 

< -0{t-to). 

Let To G [to, to + ^3] be such that 

/i(0(ro,to,xo,7(-)^'^(-))) = max /i((/)(r, to, xo, 7(0^ '^(O))- 

re[to,to-\-d3\ 

Case 1.1: If ro G (to, to + (^3], then for t = ro we have 

(10) y(0(ro,to,xo,7(-),^(-)),^o)- V(^o,to) < -^(ro - to) < 0. 

Then by the dynamic programming argument of Lemma 1 we have: 

^(^o,to)< sup [max{ max h{(t){T,to,xo,j{-),v {•))), 

^(•)eV[to,to+53] re[to,to+53] 

y((/)(To,to,xo,7(-)^^('))^^o)}]- 

We can choose v{-) G V[tQ,to-\-63] such that 

^(^o,to) < max{ max /i((/)(r, to, xo, 7(0^ '^(O))^ ^(</^(^o, to, xo, 7(0^ '^(•))^ ^0)} + e, 

re[to,to-\-53] 

and set e < f (ro - to). Since h{x) - V{x,t) < for ah (x,t) G x (0,T) we have that 

max /i(0(r,to,xo,7(-)^^('))) = ^(^(^o, to, xo, 7(0^ ^(0)) < ^(^(^o, to, xo, 7(0^ ^(0)^ ^0). 
Te[to,to+^3] 

Hence 

Q 

V{xo,to) < V((/)(ro,to,xo,7(-),^(-))^^o) + ^{tq - to). 

Since (10) holds for all v{') G V'[to,T]5 i^ will also hold for 'O(-), and hence the last argument 
establishes a contradiction. 

Case 1.2: If ro = to then for t = to -\- 63 we have that for all v{-) G V^[to,T] 

^(0(to + (^3, to, xo, 7(-). ^(O). ^0 + ^3) - V{xo, to) < -6>^3 < 0. 
Since by Lemma 1 

^(^o,to)< sup max{ max h{(f){T,to,xo,j{-),v {•))), 

V{(t){to^Ss,to,xo,j{-),v{-)),to + Ss)}, 
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then if 

V{xo,to) < sup V{(l){to + ^3, ^0, xo, 7(-)^ v{-))^to + Ss), 
we can choose v{-) ^ V[to,^o+53] ^^^^ ^^^^ 

V{xo, to) < V{^{to + ^3, to, xo, 7(-), ^(O), ^0 + (^3) + 

which estabhshes a contradiction. 
If 

V{xo,to) < sup max h{4>{r,to,xo,j{-),v {-))), 

then we can choose v{-) ^ V[tQ,to-\-63] such that 

V(xo, to) < max /i(^(r, to, xq, 7(-), ^(O)) + 

rG[to,to+53] 

or equivalently F(xo,to) < h{xo) + e, since tq = to- Based on our initial hypothesis that 
h{xo) < V{xo,to), there exists a (5 > such that h{xo) — V{xo,to) < —26. If we take e < S we 
establish a contradiction. 

Part 2. Consider an arbitrary (xo,to) G x (0,T) and a smooth VI^ : x (0,T) ^ R 
such that V — W has a local minimum at (xo,to). Then, there exists ^1 > such that for all 
{x,t) gR^ X (0,T) with |x-xoP + (t-to)^ < ^1 

{V-W){xo,to) < {V-W){x,t). 

We would like to show that 

dW dW 
max{/i(xo) - V{xo,to), ^-(xo,to) + sup mf——{xo,to)f{xo,u,v)} < 0. 
ot y^yueu ox 

Since V{x, t) > h{x) it suffices to show that ^^(xq, to)+sup^^ yinf^^^ u ^"(^o, to)f{xo, v) < 0. 
This implies that for all v G V there exists di u G U such that 

dW dW 

-^{xo,to) + -^{xo,to)f{xo,u,v) < 0. 

For the sake of contradiction assume that there exists v G V such that for all G U there exists 
> such that 

dW dW 

— (xo,to) + —{xo,to)f{xo,u,v) > 26> > 0. 

Since W is smooth, there exists 62 G (0, 5i) such that for all (x, t) G R"^ x (0, T) with |x — + 
(t - to)' < S2 

dW dW 

-gfi^^'^) + -g^ix,t)f{x,U,v) >0>0. 

Hence, following [15], for u(-) = v and any 7(-) G ^[to,T] 

dW dW 

— + _(x,t)/(x,7(-),«(-)) > ^ > 0. 

By continuity, there exists (^3 > such that \(j){t,to,xo,^{-),v{-)) — + (t — to)^ < ^2 for all 
t G [to, to + ^3]. Therefore, for ah 7(-) G Tf^^^T] 

V((/)(to + (^3, to, xo, 7(-), v{-)),to + (^3) - V{xo, to) 

> W{(l){to + ^3, to, xo, 7(-). ^(O). to + ^3) - W{xo, to) 



\~di 



= I (^((/>(t,to,xo,7(-),^(-)),t) 

^ dW 
dx 
>OSs. 



{(l){t,to, xo, -/{-), v{-)),t)f{(l){t, to, xon{-),v{-)), -/{-), v{-))^dt 
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But by the dynamic programming argument of Lemma 1 we can choose a 7(-) G '^[to,T] such 
that 

V{xo,to)> sup [max{ max /i(^(r, to, xq, 7(-), ^(O))^ 



Ss6 



> max{ max h{(l){T, to, xo,j{-), v {■))), V{(l){to ^ S3,to,xon{-),v{-)),to ^ Ss)} 

rG[to,to+53] 

S 

> V{^{to + Ss, to, xo, 7(-), O).to -^Ss)-^. 

The last statement estabhshes a contradiction, and completes the proof. □ 

3.3. Variational inequality for V. Consider the value function V defined in the previous 
section. The following theorem proposes that V is the unique viscosity solution of another vari- 
ational inequality. 



Theorem 2. V : x [0,T] ^ R 25 the unique viscosity solution of the variational inequality 

dV . . . . . . dV , 

suj 

with terminal condition V{x^T) = max{/(x), 



( - dV dV 1 

(11) max < /i(x) — y(x, t), — — (x, t) + min{0, sup inf — — (x, t)/(x, ix, i;)} > = 0, 

I at y^yueu ox J 



Proof. By Theorem 1, V{x^t) is the unique viscosity solution of (4), subject to V{x^T) = 
max{/(x), If we let H{x,p) = sup^^y'mf-^^ f{x,u^v) then, following the proof of 
Theorem 2 of [30], we have that 

H{x,p) = sup inf^p^ f{x^u^v) 
ve Vueu 

= sup inf mf_ p^ {uf{x,u,v)) 

yue U ueU 

= inf sup inf p^ f{x^u^v) 
ueu vev^^^ 

= miniZsup inf p^ f{x^u^v) 
ueu v^yueu 

= min{0, i7(x,p)}. 

Consequently, the two variational inequalities (4) and (11) are equivalent, and so V(x,t) is the 
viscosity solution of (11). □ 

Since the solution to (11) is unique [25], one could easily show that 

V(x,t) = inf sup min max{/(^(ri, t, x, 'u(-), 'i;(-))), max h((j)(r2,t, x^u(-), v (-)))}. 

4. Case study: Collision Avoidance in Air Traffic Management 

To illustrate the approach described in the previous sections, we consider a problem from 
the air traffic management area. The increase in air traffic is bound to lead to further en-route 
delays and potentially safety problems in the immediate future [32], [33]. A major difficulty with 
accommodating this expected increase in air traffic is uncertainty about the future evolution of 
flights. Therefore, the CATS research project has proposed a novel concept of operations, which 
aims to increase punctuality and safety during the flight. This concept is mainly based on impos- 
ing spatial and temporal constraints at different parts of the flight plan of each aircraft. These 
4D constraints are known as Target Windows (TW) [34], and represent the commitment from 
each actor (air traffic controllers, airports, airlines, air navigation service providers) to deliver 
a particular aircraft within the TW constraint. This commitment is known as the Contract 
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of Objectives (CoO) [27], and can be viewed as a first step towards the implementation of the 
Reference Business Trajectory envisioned by the SESAR joint undertaking [32]. 

In this section we follow the approach proposed in [28] to code the TW constraints, and 
use the reach-avoid formulation of Sections II and III, to investigate collision avoidance in the 
presence of TW constraints. For this purpose we consider a two- aircraft scenario, where each 
aircraft should respect its TW constraints, while avoiding conflict with other aircraft. 



4.1. Aircraft model. Each aircraft j = 1, A/" is assumed to have a predetermined flight plan, 
which comprises a series of way points 0(i,j) = Viij) ^ where i = l,...,Mj. 

The angle that each segment forms with the x axis and the flight path angle ^{ij) that it 

forms with the horizontal plane are shown in Fig. 1. The discrete state i stores the segment of 
the flight plan that the aircraft is currently in, and for i = 1, Mj — 1 we can define 



tan 



= tan" 



y(^+l,J)-y( 



-1 ( 



Hi,3) 



i,3) J 



where d. 



X(^i_^ij^ — X(j^^j^Y + ~ y{i,3)Y is length of the projection of its 



segment on the horizontal plane. Assume perfect lateral tracking and set s G M+ denote the the 
part of each segment covered on the horizontal plane (see Fig. 1). Based on our assumption 
that each aircraft has constant heading angle at each segment, its x and y coordinates can 

be computed by: 









+ 


COS*(ij)" 






Mi J). 







X o(i+i) 



Global Coordinate Frame 




(a) Flight plan projection on the horizontal (b) Flight plan projection on the z-s plane 
plane 



Figure 1. Flight plan projections 



To approximate accurately the physical model, the flight path angle 7J, which is the angle 
that the aircraft forms with the horizontal plane, is a control input fixed according to the angle 
^(ij) that the segment forms with the horizontal plane. If ^(ij) = the aircraft will be cruising 
(7J = 0) at that segment, whereas if it is positive or negative it will be climbing (7J G [0,7^]) 
or descending (7J G [—7^,0]) respectively. 

The speed of each aircraft apart from its type depends also on the altitude z. At each flight 
level there is a nominal airspeed that aircraft tend to track, giving rise to a function g{zj^j^). 
The dependence on the flight path angle indicates the discrete mode i.e. cruise, climb, descent, 
that an aircraft could be. For our simulations, we have assumed that at every level the airspeed 
could vary within 10% of the nominal one; this is restricted by the control input bj G [—1,1]. 
Figure 2 shows the speed-altitude profiles of the A320 (the simulated aircraft), based on the 
BADA database [35], for the different phases of flight. These curves have been computed by 
linear interpolation between the predetermined '.' points. 
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(a) CLIMB Speed-Altitude profile 



2000 4000 6000 8000 10000 
altitude (m) 



12000 14000 



(b) DESCENT Speed- Altitude profile 




2000 4000 6000 8000 10000 12000 14000 

altitude (m) 

(c) CRUISE Speed-Altitude profile 
Figure 2. Speed- Altitude profiles for the different phases of flight 



Most of the reachability numerical methods are based on gridding the state space, so the 
memory and time necessary for the computation grow exponentially in the state dimension. 
Therefore, using a full, five- or six-state, point mass model of the aircraft, like the one described 
in [36], would be computationally expensive to analyze using the existing computational tools. 
In [28], the full point mass model for the aircraft, was abstracted to a simplified one to make 
the reachability computation tractable. Motivated by the fact that aircraft track laterally very 
well, it was assumed that the heading angle remains constant at each segment. The dynamics 
of each aircraft are modeled by a hybrid automaton Hj = {Xj^Qj^ Initj^ fj^ Dorrij^Gj^ Rj) (in 
the notation of [12]), with: 
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X,. 



• continuous states Xj = [sj Zj t] G 

• discrete states i G {0, Mj — 1} = Qj. 

• initial states Initj = {(i, Sj^Zj^t) \ i = 0^ Sj = 0, Zj = zj^}. 

• control inputs Uj = [bj 7J] G [—1,1] x [—jj^^jj^]- 



• disturbance inputs 


V = 


[Wx Wy Wz]^ e 


• vector field fj : 




X Xj xUjxV ^ Xj. 








■(l + 0.16,Mz„7j; 


5j, Zj^ t) = 


^3 

i 




(1 + 0. 


• domain Dorrij = 


= {( 




Zj,t) 1 Sj < d^ij^}. 


• guards Gj{i^i 


■1) 


= {( 


Sj^ Zjj t) Sj > duj 


• reset map Rj{i^ 


i + 


l,Sj,Zj,t) = {{0,Zj,t)} 



■ Wx COS ^(^^^•) + Wy sin 



(1 + 0.16^)^(2;^-,7^)sin7^ ^w^ 



1 
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Apart from 5j, the other two continuous states are the altitude zj^ and the time t. The 
last equation was included in order to track the TW temporal constraints. As stated above, 

is the flight path angle and w is the wind speed, which acts as a bounded disturbance with 
—w < w <w^ and for our simulations we used w = 12m/ s. Since the flight path angle does 
not exceed 5°, for simplification we can assume that s'mj^ ^ 7J and C0S7J ^ 1. 

4.2. Reach- Avoid problem formulation. Target Windows represent spatial and temporal 
constraints that aircraft should respect. Following [34], we assume that TW are located on the 
surface area between two air traffic control sectors. Based on the structure of those sectors, the 
TW are either adjacent or superimposed (Fig. 3), and for simplicity we assume that there is a 
way point centered in the middle of each TW. 

Our objective is to compute the set / of all initial states at time t for which there exists a 
non-anticipative control strategy 7, that despite the wind input v can lead the aircraft j inside 
the TW constraint set at least once within its time and space window, while avoiding conflict 
with the other aircraft. In air traffic, conflict refers to the loss of minimum separation between 
two aircraft. Each aircraft is surrounded by a protected zone, which is generally thought of as 
a cylinder of radius 5nmi and height 2000ft centered at the aircraft. If this zone is violated by 
another aircraft, then a conflict is said to have occurred. To achieve this goal, we adopt another 
simplification introduced in [28]; we eliminate time from the state equations, and perform a 
two-stage calculation. 



Superimposed TW 




Adjacent TW 



Figure 3. Superimposed and Adjacent TWs 

We define the spatial constraints of a TW centered at the way point i as Rj = {d(^ij^^ + 

is adjacent, and Rj = {[di^ij) ^ Sf^^j^, d(^ij) + 5(i,j)], if the 

TW is superimposed. Let also [tj^tj] denote the time window of Rj. Then / could be computed 
as: 

Stage 1: Compute for each aircraft j the set Rj of states Xj at time tj (beginning of target 
window) from which there exist a control trajectory that despite the wind can lead the aircraft 
inside Rj at least once within the time interval [tj^tj]. But this set is the RAj{t^Rj^Aj) set, 

which was shown in Section III to be the zero sublevel set of V, which is the solution to the 
following partial differential equation 

dV dV 

m8ix{hj{xj) — V(xj,t), -—-{xjjt) + min{0, sup inf — — {xj ^t)fj{xj ^Uj ^v)}} = 0. 

Ot y^yuje Uj OXj 

The terminal condition V{xj^ij) = l{xj) was chosen to be the signed distance to the set i?^, and 
the avoid set Aj is characterized by hj{xj). This function represents the area where a conflict 
might occur, and it is computed online by performing conflict detection (see Appendix C). 
Stage 2: Compute the set / of all states that start at time t < tj and for every wind can reach 
the set Rj at time t^-, while avoiding conflict with other aircraft. Based on the analysis of Section 
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II, this is the RAj{t^ Rj^ Aj) set, that can be computed by solving 



dV , 



dV 



max{hj{xj) -V{xj,t),—-{xj,t)^sup inf -—{xj,t)f{xj,Uj,v)} = 0, 
at y^yuje Uj oxj 

with terminal condition V{xj,tj) = max{F(xj, t^), /ij(xj)}. The set Rj is defined as 

Rj={xj gR^I V{xj,t^))<0}, 

whereas Aj depends once again on the obstacle function hj{xj). The simulations for each aircraft 
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(a) Flight plans for the two aircraft case 
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(b) Flight plan projection for the two aircraft case 



Figure 4. Flight plans for the two aircraft scenario 

are running in parallel, so at every instance t, we have full knowledge of the backward reachable 
sets of each aircraft. Based on that. Algorithm 1 of Appendix C describes the implemented steps 
for the Reach- Avoid computation. 

4.3. Simulation Results. Consider now the case where we have two aircraft each one with 
a TW, whose fiight plans intersect, and they enter the same air traffic sector with a 30 sec 
difference. Fig. 4a depicts the two ffight plans and Fig. 4b the projection of the ffight plans 
on the horizontal plane. The Target Windows are centered at the last way point of each ffight 
plan. The result of the two-stage backward reachability computation with TW as terminal sets 
is depicted in Fig. 5a. The tubes at this ffgure include all the states that each aircraft could 
be, and reach its TW. We should also note that the tubes are the union of the corresponding 
sets. These sets at a speciffc time instance, would include all the states that could start at that 
time and reach the TW at the end of the horizon. Fig. 5b is the projection of these tubes on 
the horizontal plane. As it was expected, the x-y projection coincides with the projection of the 
ffight plans on the horizontal plane. This is reasonable, since in the hybrid model we assumed 
constant heading angle at each segment. Moreover, based on the speed- altitude proffles, aircraft 
ffy faster at higher levels, so at those altitudes there are more states that can reach the target. 

We can repeat the previous computation, but now checking at every time if the sets, in the 
sense described before, satisfy the minimum separation standards. That way, the time and the 
points of each set where a conffict might occur, can be detected. The result of this calculation is 
illustrated in Fig. 6a. The "hole" that is now around the intersection area of Fig. 5a represents 
the area where the two aircraft might be in conffict. 

Now that we managed to perform conffict detection, we are in a position to compute at every 
instance the obstacle function hj{xj). Since the conffict does not occur within the time interval 
of the TW, the set of the initial states that an aircraft j could start and reach the set Rj at time 
tj, while avoiding conffict with the other aircraft, should be computed. Once the aircraft hits i?2, 
it can also reach the TW within its time constraints. To obtain the solution to this reach-avoid 
problem, the variational inequality (4) should be solved. If the conffict had occurred in [tj^tj] 
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(a) Intersection of the two backward reachable tubes (b) Projection of the two backward reachable tubes 

Figure 5. Backward reachable tubes for the two aircraft scenario 
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(a) Conflict detection zone (b) Reach-Avoid tubes 

Figure 6. Conflict detection and reach- avoid computation 



equation (11) should be solved instead. One could either use numerical methods developed by 
[25], or the Level Set Method Toolbox [19], whose authors propose a way to code obstacles on 
the value function. The latter was used in this paper, and the obstacle function hj{xj) was 
dynamically determined, since at every time it is the result of the conflict detection. Fig. 6b 
shows the reach-avoid tubes at t = 25mm. As it was expected, the set of states that could reach 
the target while avoiding conflict with the other aircraft, does not include the conflict zone of 
Fig. 6a, but also some more states that would end up in this zone. This is a "centralized" 
solution, since for the safe sets shown in Fig. 6b, there exists a combination of inputs for the 
two aircraft that could satisfy all constraints, i.e. reaching their TW while avoiding conflict. It 
should be also noted that all simulations where performed on the same grid, but were running 
in parallel. Hence he have two 2D computations (one for each aircraft) instead of a 4D one, and 
hjixj) was determined at each step by comparing the obtained sets as described in Algorithm 
1. 



5. Concluding Remarks 

A new framework of solving nonlinear systems with state constraints and competing inputs 
was presented. This formulation was based on reachability and game theory, has the advantage 
of maintaining the continuity in the Hamiltonian of the system, and hence it has very good 
properties in terms of the numerical solution. The problem of reaching a desired set, in this case 
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the TW, while avoiding conflict with other aircraft was formulated as a reach-avoid problem, 
and was computed numerically by using the existing tools. 

In future work, we plan to use these reach-avoid bounds in order to perform conflict resolution 
by optimizing some cost criterion. Another issue would be to extend the proposed approach to 
formulate games in the case where the obstacle function is time and/or control dependant. 
Finally, we intend to validate our approach with fast time simulation studies using realistic 
aircraft and flight management system models, flight plans and wind uncertainty. 

Appendix A. 

A.l. Proof of Proposition 2. 

Proof. Par^l. Following [30] we flrst show that RA{r,R,A) C {x G | V{x,r) < 0}. Con- 
sider X G RA{r^ A) and for the sake of contradiction assume that V{x^ t) > 0. Then there 
exists e > such that sup^(.)^y^^^^^ max{/(^(T, t, x, 7M(-), max^^[^,T] h{4){r,t, x,j[v]{-), 

v{-)))} > 2e > 0. This in turn implies that there exists v{') G V[t^T] such that either l{(p{T^ t, x, j[v] 
{■),v{-))) > e > or there exists r G [t,T] such that h{4){r,t, x,j[v]{-),v{-))) > e > 0. 

Consider now the implications of x G RA{r^ R^ A). Equation (5) implies that there exists a 
7(-) G ^[t,T] such that for all v{-) G V[t,T]^ and so also for 'O(-), we can deflne u{-) = 7['0](-). 
Then, for this u{-) and v{-) there exists ri G [t,T] such that 0(ri, x, t, ii(-), 'O(-)) G R and for all 
T2 G [t,ri] ^(r2, t, X, ii(-), 'O(-)) ^ A. Choose the freezing input signal as 



u{s) 



1 for 5 G [t, Ti] 
forsG[ri,T] 



If we combine u{-) with li(-), we can get the input u{-) which will generate a trajectory 

' ' ' \ (^(ri,x,t,ii(-),v(-)) for rG[ri,T] 

Case 1.1: Consider flrst the case where for all 7(-) G ^[t,T] K^{'^^'^^^^l[^]{')^^{'))) > e > 0. 
For s = T we have that 

4){T,x,t,u{-),v{-)) = (j){Ti,x,t,u{-),v{-)). 

Since x G RA{r, R, A), we showed before that ^(ri, x, 'u(-), 'O(-)) G R, i.e. /(0(ri, x, t, 'u(-), 'O(-) 
)) < 0. So from we have that /(^(T, x, t, ii(-), 'O(-))) < 0. Since u{-) = 7['0](-) is already non- 
anticipative, and a non-anticipative strategy for u{-) can be designed, u{-) will also be non- 
anticipative. Therefore, the previous statement establishes a contradiction. 

Case 1.2: Consider now the case where for all 7(-) G ^[t,T] there exists r G [t,T] such that 
h{(l){r^ X, "y[v]{-)^v{-))) > e > 0. Since we showed that for all r G [t, n], ^(r, t, x, i^(-), v{-)) ^ A, 
we can conclude that for all r G [t^ri] 

h{^{r,x,t,u{-),v{-)))<0. 

If r G [ri , T] , then we have that 

4){T,x,t,u{-),v{-)) = 0(ri,x,t,ii(-),^(-))- 

So /i(^(r, X, t, ii(-), 'O(-))) = /i(0(ri, X, t, li(-), 'O(-))) < 0. Hence, for all r G [t,T] we have that 
/i(^(r, X, 'u(-), 'O(-))) < 0. Since in Case 1.1, u{-) was shown to be non-anticipative, we have a 
contradiction. 

Part 2. Next, we show that {x G M"' | V{x,r) < 0} C RA{r,R,A). Consider {x,t) such 
that V(x, < and assume for the sake of contradiction that x ^ RA{r, R^ A). Then for all for 
all 7(-) G ^[t,T] there exists v{-) G V[t,T] such that for all ri G [t, T] either (/)(ri, t, x, 7(-), '^(•)) ^ R 
or there exists r2 G [t,ri] such that (/)(r2, t, x, 7(-), 'L'(-)) G A. 
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Following the analysis of [30], consider that the strategy 7(-) G ^[t,T] is extracted from 7(-) G 
^[t,T]^ and choose the v{-) that corresponds to that strategy. In [30], was proven that the set of 
states visited by the augmented trajectory is a subset of the states visited by the original one. 
We therefore have that for all ri G [t, T] 

(12) ^{rux,t,-f[v]{-),v{-)) ^(ri,x,t,7[{)](-),i)(-)) ^ 
and also Vr2 G [t, ti] 

(13) (/)(r2,x,t,7[i)](-),^(-)) eA=^ ^{T2,x,t,^[v]{-),v{-)) G A. 

By (12), (13) we conclude that there exists a (5 > such that either for all ri G [t,T] 

(14) /(^(ri,x,t,7[^](-),^(-)))>^>0, 
or for some r| G [t, ri] 

(15) /i(<^(t2*, x,t,7[i)](-), «(•))) ><5>0. 

Since F(x, t) < 0, then for all e > there exists a non-anticipative strategy 7(-) G ^[t,T] such that 

sup^(.)^y^^^^^, max{/(^(T,t,x,7[v](-),v(-))),max^^[^,T] ^(^(T,t,x,7[v](-),v(-)))} < e. Hence for 

ah v{-) G V[,,T], /(^(T,t,x,7[^](-),^(-))) < e and for ah r G [t,T], /i(^(r, x, 7[^](-), ^(O)) < e. 
For 'u(-) = 'O(-) the last argument implies that 

/(^(T,x,t,7[i)](-),^(-))) <e, 

and there exists T2 G [t, ti] 

/i(^(r2,x,t,7[i)](-),^(-))) <e. 

2, the last state 

Appendix B. 

B.l. Proof of Lemma 1. 



If we choose e = |, the last statements contradict (14), (15) and complete the proof. □ 



Proof. Following [15] we can define 

W{x^t)= inf sup [max{ max /i(^(r, x, ii(-))), ]/(^(t + a, t, x, ix(-)), t + a)}] . 

We win then show that for ah e > 0, V{x,t) < W{x,t) + 2e and V{x,t) > W{x,t) - 3e. Then 
since e > is arbitrary, V{x^t) = W{x^t). 

Case 1: V{x^t) < W{x,t) + 2e. Fix e > and choose 7i(-) G Ff^^^+c^] such that 

W{x,t) > sup [max{ max /i(^(r, t, x, 7i(-), 'Ui(-))), 

vii-)eV[t,t+c.] re[t,t+a] 

V(^(t + a,t,x,7i(-),vi(-)),^ + Q^)}] - 
Similarly, choose 72(-) G F[^^ck 7^] such that 

y(^(t + a,t,x,7i(-),^i(-)),^ + c^) > 

sup max{/((/)(T,t + a, (/)(t + a, t, x, 7i(-), vi(-)), 72(0, ^2(-)))^ 

^^2(-)eV[t + c.,T] 

max /i(^(r, t + a, ^(t + a, t, x, 7i(-), Vi(-)), 72(-). ^2(-)))} - 

rG[^+Q!,T] 

For any v{-) G V[^^t] we can define '^i(-) G V[^^t+Q,] and '^2(0 ^ V[^+a,T] such that Vi{r) = v{r) 
for all r G [t, t + a) and i'2(t) = v{r) for all r G [t -\- a^T]. Define also 7(-) G ^[t,T] by 



7[v](r) 



f 7ibi]M if TG [t.t^a) 
1 72[^2](r) if TG [t + a,T]. 



It easy to see that 7 : V[^^t] ^ ^^,T] is non-anticipative. By uniqueness, ^(r, x, 7(-), 'u(-)) = 
^(r,t,x,7i(-),vi(-)) if ^ ^ [t,t + a), and ^(r, x, 7(-), ^(•)) = ^(r, t + a, ^(t + a, t, x, 7i(-), vi(-)), 
72(0, ^2(0) ifr G [t + a,T]. 
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Hence, 

W{x^t) > sup sup max { max /i((/)(r, 7i(-), i'i(-))), 

/((/)(T,t + a,(/)(t + a,t,x,7i(-),^i(-)),72(-),^2(-))), 

max /i(^(r,t + + a,t,x,7i(-),vi(-)),72(-).^2(-)))} - 2e 

rG[t+a,T] 

> sup max{/(^(T,t,x,7(-),v(-))), m^y^ h{^{T,t,x, v {■)))}- 2e 

> V(x,t) - 2e. 
Therefore, < W{x,t) + 2e. 

Case 2: V{x^t) > W{x^t) — 3e. Fix e > and choose now 7(-) G ^[t,T] such that 

(16) V{x,t)> sup max{/(0(T,t,x,7(-),v(-))), max /i(^(r, x, 7(-), ^lO))} - e. 

By the definition of W{x^ t) 

W{x^t) < sup [maxj max /i(^(r, x, 7(-), 'u(-))), V(^(t + a, t, x, 7(-), 'u(-)), t + . 
Hence there exists a vi{-) G V[t^t-\-a] such that 

(17) W{x,t) < max{ max /i(^(r, t, x, 7(-), ^i(-))), V'(0(t + a, t, x, 7(-), vi(-)), ^ + c>^)} + e. 

^ Te[t,t+a] 

Let 'O(r) = i'i(r) for all r G [t,t + a) and v{r) = '^'(r) for all r G [t + a,T]. Let also G r[^^Q!,T] 
to be the restriction of the non-anticipative strategy 7(-) over [t+a, T]. Then, for all r G [t+a, T], 
we define 7'['^'](t) = 7['0](r). Hence 

V(0(t + a,t,x,7(-),^i(-)),^ + c^) < 

sup max {/((/)(T, t + a, (/)(t + a, t, x, 7(-), vi(-)), 7'(-)^ ^'(O))^ 

max /i(^(r,t + a,0(t + a,t,x,7(-),vi(-)),7'(-).^'(-)))}^ 

rG[^+a,T] 

and so there exists a '^2(0 ^ '^[t-\-a,T] such that 
(18) 

V((/)(t + a,t,x,7(-),vi(-)),t + a) < max{/((/)(T,t + a,^(t + a,t,x,7(-),vi(-)),7 (•),V2(-)))^ 

max /i(^(r, t + a, ^(t + a, t, x, 7(-), vi(-)), V (•). ^2(-)))} + 

rG[t+Q;,T] 



We can define 

^ ^"1 ^2(r) ifrG[t + a,T] 

Therefore, from (17) and (18) 

W{x, t) < max {/((/)(T, t, x, 7(-), ^(0)), max /i((/)(r, t, x, 7(-), ^(0))} + 2e, 

rG[^,T] 

which together with (16) implies V{x^t) > W{x^t) — 3e. □ 
B.2. Proof of Lemma 2. 

Proof. Since and h are bounded, V is also bounded. For the second part fix x,x G and 
t G [0,T]. Let e > and choose 7(-) G ^[t,T] such that 

V(x,t)> sup max max{/((/)(T, t, x, 7(-), 'L'(-))), /i((/)(r, t, x, 7(-), 'L'(-)))} — e. 
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By definition 

V{x, t) < sup max max{/(^(T, t, x, 7(-), v{'))), h{(t)(T, t, x, 7(-), ^(O))}- 

We can choose 'O(-) G V[t,T] such that 

V{x, t) < max max{/((/)(T, t, x, 7(-), ^(O)), ^(^(t, t, x, 7(-), ^(O))} + 

rG[^,T] 

and hence 

V{x, t) - t) < max max{/((/)(T, t, x, 7(-), v{-))), h{^{r, t, x, 7(-), ^(O))} 

- max max{/((/)(T, x, 7(-), ^(O)), K^{r, t, x, 7(-)^ ^(O))} + 2e. 

re[t,T] 

For ah r G [t,T]: 

|(/)(r,t,x,7(-),^(-))-</>(^,^,^,7(-),^(-))l = 

I (x - x) + ^ X, 7(-), ^(O)) - / W^, l{-).^{-)))]ds\ 

< |x-x|+ / |/((/)(5,t,x,7(-),^(-)))-/(^(5,^,^,7(-),^(-)))l^^ 
< 



\x - x\^Cf |0(5,t,x,7(-),^(-)) - ^(s,t,x,7(-),^(-))l^5, 



where Cf is the Lipschitz constant of /. By the Gronwall-Behman Lemma [37], there exists a 
constant > such that for all r G [t, T] 

|(/)(r,t,x,7(-),^(-))-^(T,^,^,7(-),^(-))l <Gx|^-^|. 
Let To G [t, T] be such that 

/i((/)(ro, t, X, 7(-), = max /i(0(r, t, x, 7(-), ^(O))- 

re[t,T] 

Then 

l/(x, - F(x, < max{/((/)(T, t, x, 7(.), ^(O)), ^ Wro, v{-)))} 

-max{/((/)(T,t,x,7(-),^(-))),^Wro,t,x,7(-),^(-)))} + 2e. 

Case 1. /((/)(T,t,x,7(-),^(-))) > M</>(^o, 7(-), ^(O)) 

y(x,t)-F(^,t)</((/)(T,t,x,7(-),^(-))) 

- max{/((/)(T, t, X, 7(-), v{-))), h{^{ro, t, x, 7(-), ^(O))} + 2e 

< /((/)(T, t, X, 7(-), ^(O)) - X, 7(-), ^(O)) + 26 

< aC3,|x-x| +2e. 

Case 2. /((/)(T, t, x, 7(-), ^)(-))) < M^(^o, 7(-), ^(O)) 
y(x, t) - F(x, t) < /i((/)(ro, t, X, 7(-), ^(O)) 

- max{/((/)(T, t, X, 7(-), ^)(-))), K^{t^, t, x, 7(-), ^(O))} + 2e 

< /i((/)(ro,t,x,7(-),^(-))) - K^{TQ,t,x,%-),v{-))) + 2e 
<C/,C,|x-x|+2e. 

So in any case V(x,t) — V(x,t) < max{QC/i}Ca;|x — x| + 2e. The same argument with the 
roles of X, x reversed establishes that V(x,t) — V(x,t) < max{C^C^}Ca;|x — x| + 2e. Since e is 
arbitrary, 

|F(x,t) - V{x,t)\ < max{aC4C,|x - x|. 



18 



K. MARGELLOS AND J. LYGEROS 



Finally consider x G and t, t G [0, T]. Without loss of generality assume that t <t. Let e > 
and choose 7(-) G '^[t,T] such that 

V{x, t) > sup max max{/(^(T, t, x, 7(-), v{-))), h{(j){r, t, x, v{-)))} - e 



> max max{/(0(T, t, x, l{'),v{'))), h{(j){T, t, x, l{'),v{')))} - e 

re[t,T] 



By definition, 



V{x,i) < sup max max{/(0(r, £, x, 7(-), '?;(•))), ^, 7(-)^ '^(O))}- 

So we can choose v{-) ^ '^[i,T] ^^^^ ^^^^ 

< max max{/((/)(T,t,x,7(-),i)(-))),/^(0(^,^^^,7(-)^^(')))} + 

re[t,T] 

where 7 G r^^j.] is the restriction of 7(-) over [t, T]. Then, for all r G [t, T], we define j[v]{r) = 
7['u](r), and 'O(r) = 'i;(r+t — t). By uniqueness, for all r G [t, T] we have that (/)(t, t, x, 7(-), = 
(/)(r + t-t,t,x,7(-),^(-)). 

l/(x, t) - V(x, i) > max max{/(0(T, t, x, 7(-), vf-))), hU{r, t, x, j(-), v(-)))} 

re[t,T] 

- max max{/((/)(T,t,x,7(-),i)(-))),/i(0(^,^^^,7(-)^^(')))} - 2e. 

re[t,T] 

Case 1. /(0(r,t,x,7(-),'O(-))) > niax^^^^^^] /i(0(r, t, x, 7(-), 

t) - i) > max max{/(0(r, t, x, 7(-), ^(O)), 7(-), ^(O))} 

re[t,T] 

-/((/)(T,t,x,7(-),^(-)))-26 

> /(0(T, t, X, 7(-), ^(O)) - i 7(-), ^(O)) - 26 

= /(0(T, t, X, 7(-), ^(O)) - K^{T + t-i, t, x, - 26 

> -CiCf\T -T -t + i\-2e 
= -CiCf\i-t\-2e, 

where Ci is the Lipschitz constant of /. 

Case 2. l{(l){T,i,x,^{-),v{-))) < max^^[£^^] /i(0(r, f, x, 7(-), 

V{x, t) - V{x, i) > max max{/((/)(r, t, x, 7(.), ^(O)), t, x, 7(-), ^(O))} 

rG[t,T] 

- max /i((/)(T,t,x,7(-),i)(-))) - 2e 

re[t,T] 

> max /i(0(r, t, X, 7(-), t'(-))) — max /i((/)(r, x, 7(-), 'D(-))) — 2e. 

-^e[t,T] re[i,T] 

Let To G [t, T] be such that 

/i(0(ro,t,x,7(-),i)(-))) = max /i(0(r, t, x, 7(-), 

re[t,T] 



Then 



y(x, t) - V{x, i) > max /i((/)(r, x, 7(.), v{-))) - /i((/)(ro, x, j{'),v{-))) - 2e 

rG[t,T] 

> M(/)(ro, t, X, 7(.), ^(O)) - M^(ro, 7(-), ^(O)) - 2e 

= /i((/)(ro, t, X, 7(-), ^(O)) - H^iro + t-i t, x, j{-),v{-))) - 2e 

>-ChCf\ro-To-t + i\-2€ 

= -ChCf\t-t\-2e, 
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where Ch is the Lipschitz constant of h. In any case we have that 

V{x,t) -V{xJ) > -md.x{Ci,Ch}Cf\t-t\ -2e. 

A symmetric argument shows that V{x, t) — V{x^ i) < max{(7/, Ch}Cf\i — t| + 2e, and since e is 
arbitrary this concludes the proof. □ 

Appendix C. 

The fohowing algorithm summarizes the steps of the Reach- Avoid computation described in 
Section IV. For simplicity, we have assumed that the TW do not overlap. 



Algorithm 1 Reach- Avoid computation 



1: Initialization. Set: 
T = maxj tj, for j = 1, 
to = miuj tj, for j = 1, A^, 

V{xj,tj) = l{xj) , 

Rj={xj I V{x,rtj)<0} . 
2: for t = T until to do 
3: while j is in the sector 

4: if tj <t< tj 

Solve + min{0,sup^^yinf^.et/,. ^,{xj,t)fj{xj,Uj,v)} = Q. 

5: for all i 7^ j in the sector 

Cji = {xj I i, j are in conflict}. 
6: for all j G Cji 

Define hji{xj) such that Aji D Cji is a box. 

Let Aji = {xj I hji{xj) > 0}, and Aj = \J.^j Aji, 

hj = max^r^j hjii^Xj^, 

V{xj,t) = max(/ij, V{xj,t)). 

else 

7: end for 

8: end for _ 

9: V{Xj,tj) = l^(xj,t). 

10: else if t < tj 

Solve +sup^^yinf^^.et/,. §^{xj,t)fj{xj,Uj,v) =0. 

Repeat steps 5 — 8 with V instead of V. 
11: end if 
12: end while 
13: end for 
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